Inducement of population sparsity

The pioneering work on parameter orthogonalization by Cox and Reid is presented as an inducement of abstract population-level sparsity. This taken a unifying theme for this article, in which sparsity-inducing parameterizations or data transformations are sought. Three recent examples framed light: sparse covariance models, the construction factorizable elimination nuisance parameters, inference high-dimensional regression. Strategies problem exact approximate sparsity appear to be context-specific may entail, instance, solving one more partial differential equations specifying parameterized path through transformation parameterization space. Open problems emphasized. Les travaux de et sur l'orthogonalisation des paramètres sont présentés comme un moyen d'induire une certaine parcimonie à l'échelle la population. Cela se conçoit thème unificateur cet dans lequel recherchées paramétrisations ou données engendrant caractère parcimonieux. D'ailleurs, trois exemples récents examinés sous angle: les éparses modèles covariance, l'élimination nuisibles par factorisables, l'inférence en régression haute dimension. Le problème d'induction du épars, soit exacte approximative, requiert stratégies qui dépendent spécificité contexte. Elles peuvent, exemple, reposer résolution d'une plusieurs équations aux dérivées partielles imposer spécification d'un chemin paramétrique travers l'espace paramétrisations. Quelques problèmes ouverts discutés. Sparsity, existence many zeros near-zeros some domain, plays at least two roles statistical theory: aid interpretation restrain estimation error associated with multitudinous parameters. ideas motivated primarily latter, low-dimensional interest encapsulating relevant aspects interpretation. In contexts, there natural interpretable notion sparsity, challenge now rather routine task estimator that exploits structure give appropriate guarantees. See, e.g., Wainwright (2019) extensive account covering numerous examples. present article barely concerned estimators other sample quantities. Its contribution explore idea certain forms abstract, systematically induced, seek unification isolated perspective. precursor, although not inducement, appears paper & (1987). four principal considered fall broadly into categories: induced reparameterization data. exposition here follows separation, it plausible approaches connected. To avoid repetition, only essential each case presented, from synthesis attempted. Direct use likelihood function often produces misleading estimates parameters when dimension vector similar order magnitude number independent observations, typically suboptimal even moderately One resolution, implicit invocation (Cox Reid, 1987), prior interest. Let ψ λ represent respectively, parametric model. Where i ( , ) denotes corresponding off-diagonal block Fisher information matrix, said globally orthogonal if = 0 all locally equality holds particular values. implication orthogonality maximum behaves “almost if” were fixed its true value sense ^ − O p n 1 / 2 )-neighbourhood value. Here, unconstrained maximizes over constrained space . previous statements assume fixed. Parameter also enables higher-order via simple modification profile log-likelihood without specification ancillary complement (Barndorff-Nielsen, 1983; 1987). Covariance matrices their inverses encountered throughout classical multivariate analysis, almost always estimated, necessitates assumption extend procedures settings. More precisely, requirement precision matrix consistent spectral norm tends infinity effective size under suitable scaling condition. notional asymptotic regime theoretical device, means studying probabilistic behaviour > n. Spectral-norm consistency, while achievable constraint, assumptions made valid adequate approximation. motivates search models sparse. particular, raises question, we refer henceforth Q ∗ : given (relevant) model obviously any can deduced? An answer would enable achieve maximal transformed scale, could achieved exploiting before transforming conclusions back scale Battey established guarantees such procedure assuming known reliably does address parameterizing thereby estimating using device analogous proposed Box (1964) has been suggested Peter McCullagh unpublished communication. Figure was obtained generating 100 realizations random, L taking support α random samples s index set { … + }. done different values indicated 1. nonzero basis coefficients then drawn standard normal distribution, latter aspect irrelevant far ∑ concerned: distribution have used instead. indicates priori unexpected eigenvectors eigenvalues translates inverse. Specifically, exists permutation P W ⊤ where consists potentially large, dense otherwise diagonal. provided analysis expressed terms random-matrix key conclusion considerably than extended another class Rybak (2021), but scope further progress seems substantial. Most notably, earlier contains brief discussion how traverse paths representation. These chosen, following McCullagh's proposal, pass inverse parameterizations, well logarithmic parameterization. transformations, viewed technically convenient way assessing discrete compatibility Any yielded nonphysical final analysis. situation context different, goal interest—a line argument Suppose X b responses blocks individuals, variables. simplest realistic example T C outcome variable pairs homozygotic twins, twin pair chosen receive treatment denoted untreated control effect presence pair-specific arising inability unwillingness process detail. may, genetic differences between pairs. Conditional (Bartlett, 1936, 1937) marginal (Fraser, 1968) special cases pa ; x replaced product conditional probability functions, evaluated A leading encompassing form Equation (3.1) (1972) evade baseline hazard proportional hazards matched-comparison setting, found making density mass f S free so ∏ ). Such factorizations, partial-likelihood component need exist, question whether useful versions available. (Lindsay, 1980) solidifies ideas. Similar based distributions (1958), Hinkley (1974), Barndorff-Nielsen (1994). Example 1.Suppose exponentially distributed rates ψ, respectively. depend Thus identically likelihood-based connection derivative respect zero Thus, ∇ evaluation points corresponds Battey, Lee (2022) deducing transformation. integro-differential constructed Laplace transform convertible contexts. approach inducing outlined (Battey 2022) stemmed attempt formulate linear regression Section 3.1. Sparsity data, linearity Each coefficient treated turn interest-respecting sought (the near-zeros) β v or, rather, emerged had normality made. bias ˜ decays rate ), that, ignoring Wald-based accurate There strong connections regression, notably Zhang (2014) van Geer al. (2014). conceptual distinction aligns these works above induces contributions it. confidence intervals part broader inferential framework uncertainty sets emphasized (1968, 1995), Snell (1974, 1989), (2017). short synthesize inducement: yield components objects. contrast large body objects assumed subsequently imposed penalization thresholding. 2.2, model, What formalization contexts? If interpretability accepted immaterial permitted paths. How operationalized? most formulation highlighted 3.1? allow weakly parameter. should adequacy assessed? Are systematic routes fruitful factorizations generally, beyond matched comparison five posed (1975) remained open since, except modest (2022). 3.2 composite observed relied Is broad enough encompass both Sections 3.1 3.2? relation (4) (5): recoverable direct application normal-theory unknown variance σ minimal sufficient statistic squares residual sum divided degrees freedom. estimate too small, particularly covariates appreciable relative observations. By sufficiency independence joint factorizes y | reliable construct likelihood. point considering recover correct answer, do seamless theory, applied challenging situations. See Fraser, Lin (2018) vein. Likelihood theory important enlightening geometric (including cases) fit discussion? Can data-based reparameterizations (1987)? Some informal remarks (2022), Fraser (1964), spaces local location Polson devised quantifying extreme theory. unclear. unified Fisherian smaller size, yet operational converse true. inevitable, take unusual exemplified article. I am grateful anonymous referees helpful feedback UK Engineering Physical Sciences Research Council (EP/T01864X/1).